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What is claimed is: 

1 . A method for video analysis and content extraction, comprising: 

scene analysis processing of at least one video input stream; 

object detection and tracking for each scene, and; 

split and merge behavior analysis for event understanding. 

2. The method as claimed in claim 1, further comprising: 
storing behavior analysis results. 

3. The method as claimed in claim 2, wherein the behavior analysis results are stored in 
a database. 

4. The method as claimed in claim 2, wherein the behavior analysis results are stored in 
at least one video output stream. 

5. The method as claimed in claim 1, wherein the scene analysis processing further 
includes: 

scene change detection. 

6. The method as claimed in claim 1, wherein the scene analysis processing further 
includes: 

camera calibration. 

7. The method as claimed in claim 1, wherein the scene analysis processing further 
includes: 
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scene geometry estimation. 

8. The method as claimed in claim 1, wherein the object detection and tracking step 
further comprises: 

identifying a split behavior. 

9. The method as claimed in claim 8, wherein the split behavior includes an object 
splitting into two or more objects. 

10. The method as claimed in claim 1, wherein the object detection and tracking step 
further comprises: 

identifjang a merge behavior. 

11. The method as claimed in claim 10, wherein the merge behavior includes two or more 
objects merging into a single object. 

12. The method as claimed in claim 1, wherein the object detection and tracking step 
further comprises identifying zero or more split behaviors and zero or more merge 
behaviors. 

13. The method as claimed in claim 12, wherein the split behaviors and merge behaviors 
are combined to model complex behaviors. 

14. The method as claimed in claim 13, wherein the complex behaviors include package 
drop off, package exchange, crowd formation, crowd dispersal, people entering 
vehicles, and people exiting vehicles. 

15. The method as claimed in claim 1, wherein the behavior analysis step further 
comprises generating a directed graph including zero or more split behavior states and 
zero or more merge behavior states. 
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16. The method as claimed in claim 15, wherein the behavior analysis step further 
comprises generating a hidden Markov model including the directed graph. 

17. The method as claimed in claim 4, wherein the results are stored as metadata. 

18. The method as claimed in claim 8, wherein the split behavior identification applies 
the formula: 



19. The method as claimed in claim 10, wherein the merge behavior identification applies 
the formula: 



Ai'^'CliA,'^' UAj''' )#0and 

20. The method as claimed in claim 13, wherein the complex behaviors are categorized as 
one of simple, compound, and chain behaviors. 

21. An apparatus for video content analysis comprising: 
a processor for receiving and transmitting data; and 

a memory coupled to the processor, the memory having stored therein instructions 
causing the processor to perform scene analysis processing of at least one video input 
stream, detect and track objects for each scene, and analyze split and merge behaviors for 
event understanding. 
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22. The apparatus as claimed in claim 21, wherein the memory further comprises 
instructions causing the processor to store analysis results in at least one video output 
stream. 

23. The apparatus as claimed in claim 22, wherein the memory further comprises 
instructions causing the processor to store the results as metadata. 

24. The apparatus as claimed in claim 21, wherein the memory further comprises 
instructions causing the processor to perform at least one of scene change detection, 
camera calibration, and scene geometry estimation. 

25. The apparatus as claimed in claim 21, wherein the instructions causing the processor 
to detect and track objects for each scene further comprises identifying zero or more 
split behaviors and zero or more merge behaviors. 

26. The apparatus as claimed in claim 25, wherein the instructions causing the processor 
to identify zero or more split behaviors and zero or more merge behaviors further 
comprises combining the split and merge behaviors to model complex behaviors. 

27. The apparatus as claimed in claim 21, wherein the instructions causing the processor 
to analyze split and merge behaviors further comprises generating a directed graph 
including zero or more split behavior states and zero or more merge behavior states. 

28. The apparatus as claimed in claim 27, wherein the instructions causing the processor 
to analyze split and merge behaviors further comprises generating a hidden Markov 
model including the directed graph. 

29. The apparatus as claimed in claim 25, wherein the instructions causing the processor 
to identify zero or more split behaviors includes the formula: 
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30. The apparatus as claimed in claim 25, wherein the instructions causing the processor 
to identify zero or more merge behaviors includes the formula: 
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